Skip to content

[ty] Intersection simplifications with gradual generic specializations#26365

Draft
sharkdp wants to merge 8 commits into
mainfrom
david/more-gradual-generics-simplifications
Draft

[ty] Intersection simplifications with gradual generic specializations#26365
sharkdp wants to merge 8 commits into
mainfrom
david/more-gradual-generics-simplifications

Conversation

@sharkdp

@sharkdp sharkdp commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

In isinstance(..., C) and TypeIs[..] narrowing, we currently intersect with the top materialization Top[C[Unknown]] of a generic type C. If we want to relax that to an intersection with C[Unknown] directly, then we will not just see intersections like C[P] & C[Unknown] in the if branch, we'll also see intersections like C[P] & ~C[Unknown] in the else branch. If we don't simplify/transform these intersections further, we are currently unable to infer that the bottom materialization of something like list[str] & ~list[Unknown] is Never1. So here, we implement the following intersection transformation:

# For C a co-, contra-, or invariant generic class:
C[P] & ~C[Unknown] = C[P] & ~Bottom[C[Unknown]] & Unknown

In this form, due to the intersection with a plain Unknown, it is now obvious to ty that the gradual type extends all the way down to Never.

I realize that the type on the right hand side doesn't look better, but it is much easier to reason about. Also, for some specific C's, the ~Bottom[C[Unknown]] may simplify to object, so it falls out of the intersection. For example, I believe we could further simplify type[P] & ~type[Unknown] = type[P] & Unknown in a follow-up.

In addition, we also implement these transformations that we have derived before:

Co[P] & Co[Unknown] = Co[P & Unknown]
Contra[P] & Contra[Unknown] = Contra[P | Unknown]

Ecosystem

A few new diagnostics since we remove a TODO type for intersections of type[..]. All new diagnostics are true positives (they all have suppression comments for other type checkers).

Test Plan

Footnotes

  1. We might actually be able to infer that if we explicitly asked for the bottom materialization of that type. But when accessing attributes/methods on the type, we don't explicitly perform that operation. Instead, we iterate over the intersection elements and combine the attribute types.

@sharkdp sharkdp added the ty Multi-file analysis & type inference label Jun 25, 2026
@astral-sh-bot

astral-sh-bot Bot commented Jun 25, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 94.47%. The percentage of expected errors that received a diagnostic held steady at 89.19%. The number of fully passing files held steady at 95/134.

@astral-sh-bot

astral-sh-bot Bot commented Jun 25, 2026

Copy link
Copy Markdown

Memory usage report

Memory usage unchanged ✅

@astral-sh-bot

astral-sh-bot Bot commented Jun 25, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-return-type 2 0 0
unused-type-ignore-comment 0 2 0
invalid-argument-type 1 0 0
Total 3 2 0

Raw diff:

psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/rows.py:145:12 error[invalid-return-type] Return type does not match returned value: expected `type[tuple[object, ...]] & type[NamedTupleLike]`, found `<class 'Row'>`
+ psycopg/psycopg/types/composite.py:514:12 error[invalid-return-type] Return type does not match returned value: expected `type[tuple[object, ...]] & type[NamedTupleLike]`, found `<class '<unknown>'>`

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/quilt.py:856:58 warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
- static_frame/core/quilt.py:862:46 warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ static_frame/test/unit/test_quilt.py:2111:45 error[invalid-argument-type] Argument to bound method `Quilt._axis_tuple` is incorrect: Expected `(type[tuple[object, ...]] & type[NamedTupleLike]) | None`, found `<class 'tuple'>`

Full report with detailed diff (timing results)

@sharkdp sharkdp force-pushed the david/more-gradual-generics-simplifications branch 3 times, most recently from a15c984 to 13da00f Compare June 26, 2026 13:00
Comment on lines +413 to +414
static_assert(is_equivalent_to(type[P] & ~type[Any], type[P] & ~type[Never] & Any))
static_assert(is_equivalent_to(~type[Any] & type[P], type[P] & ~type[Never] & Any))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type[Never] is probably Never? So ~type[Never] could be removed from the intersection. It seemed orthogonal to the PR here, so I left it for now.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type[Never] is probably Never?

Yup

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sharkdp sharkdp force-pushed the david/more-gradual-generics-simplifications branch from 13da00f to 7143abd Compare June 29, 2026 13:12
@sharkdp sharkdp marked this pull request as ready for review June 29, 2026 13:39
@sharkdp sharkdp requested a review from a team as a code owner June 29, 2026 13:39
@astral-sh-bot astral-sh-bot Bot requested a review from dhruvmanila June 29, 2026 13:39
@sharkdp sharkdp force-pushed the david/more-gradual-generics-simplifications branch from ecbba11 to b534fbe Compare June 29, 2026 18:25
@sharkdp sharkdp marked this pull request as draft July 1, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants